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RETURN  ON  INVESTMENT  OF  NETWORK  DESIGN  EXCHANGE 

(NDEX) 

ABSTRACT 

The  purpose  of  this  MBA  Project  was  to  investigate  and  provide  a  calculated  Return 
on  Investment  of  a  new  ship  alteration  design  process  tenned  Network  Design  Exchange 
(NDEX).  The  project  evaluated  the  differences  between  NDEX  and  the  current  system  being 
utilized.  Costs  associated  with  the  implementation  and  use  of  NDEX  were  contrasted  to 
costs  of  the  status  quo  design  process  to  determine  cost  savings  and  benefits  and  compute  the 
return  on  investment  (ROI)  of  NDEX. 

NDEX  changes  the  way  C4I  equipment  is  configured  for  installation  onboard  United 
States  Navy  ships.  It  is  being  beta  tested  on  several  different  classes  of  ships. 

The  current  design  process  although  effective,  is  highly  inefficient.  It  is  an  antiquated 
system  that  is  labor  intensive  and  uses  manual  drawings  to  design  compartment  utilization 
and  ensure  the  proposed  equipment  to  be  installed  will  fit  in  the  correct  space  allotted.  The 
current  process  also  takes  a  considerable  amount  of  time,  upwards  of  18  months  from  start  to 
finish,  and  requires  many  unnecessary  steps  to  accomplish. 

The  proposed  system,  NDEX,  is  based  on  commercially  available  software  that  will 
allow  the  creation  of  ship’s  drawings  on  laptops  using  computer  aided  design  (CAD) 
applications  and  class  design  baseline  (CDB)  drawings  that  show  how  a  ship  was  configured 
at  construction.  With  NDEX,  design  cycle  time  can  be  shortened  to  weeks  if  time  criticality 
is  necessary.  NDEX  allows  a  considerable  amount  of  time  to  be  saved  by  eliminating  a  lot  of 
the  redundant  efforts  in  the  current  process.  With  NDEX,  a  laptop  computer  is  brought  to  the 
site  visit  and  drawings  are  made  during  the  visit  based  on  the  class  design  baseline  (CDB) 
drawings.  A  library  of  components  are  available  that  are  click  and  paste  which  can  be  used 
to  add  equipment  that  is  in  the  space  but  not  shown  on  the  CDB  drawings.  The  value-added, 
or  notional  benefit,  and  the  avoided  cost  realized  by  the  investment  in  this  new  NDEX 
process  is  the  focus  of  this  ROI  measurement. 
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I.  INTRODUCTION 


A.  PURPOSE 

This  report  modifies  and  applies  the  Jones-Bigham-Goudreau  notional  return  on 
investment  (JBG  NROI)  formula  to  a  design  process  system  known  as  Network  Design 
Exchange  (NDEX)  for  Program  Executive  Office  Command,  Control,  Communication, 
Computers,  and  Intelligence  and  Space  New  Ship  Construction  (PEO  C4I  SC)  at  Space 
and  Naval  Warfare  Systems  Command  (SPAWAR).  Specifically,  the  data  used  in  the 
NROI  formula  is  applied  to  the  management  of  design  processes  inherent  in  ship 
alterations  and  upgrades  of  radio  control  suite  (RCS)  equipment.  This  report  provides  a 
detailed  analysis  of  the  application  of  one  approach  to  ROI  in  the  public  sector,  with 
specific  emphasis  on  modifying  and  applying  previous  work  perfonned  cooperatively  by 
Naval  Postgraduate  School  students  and  members  of  the  staff  with  consultants  to  the 
SPAWAR  PEO  C4I  Space  Command  to  an  existing  management  decision. 


B.  BACKGROUND 

Representatives  from  SPAWAR,  San  Diego  approached  Dr.  Lawrence  R.  Jones, 
RADM  George  R.  A.  Wagner  SPAWAR  PEO  C4I  Space  Professor  of  Public 
Management,  Graduate  School  of  Business  and  Public  Policy  at  the  Naval  Postgraduate 
School  (NPS)  regarding  this  project  in  hope  of  detennining  a  return  on  investment  in 
support  of  developing  NDEX.  In  February  2005,  LCDR  Douglas  W.  Harold  and  LCDR 
Aaron  S.  Traver  (the  authors)  volunteered  for  this  project  for  their  Student  MBA  Project 
with  an  anticipated  completion  date  of  November  2005.  Since  February,  three 
teleconferences  have  been  held  with  SPAWAR  and  Northrop  Grumman  representatives, 
including  Mr.  Tom  Underwood,  Mr.  Ken  Okamura,  Mr.  Jay  Johnson,  and  Mr.  Edward 
Dalton.  On  March  11th,  LCDR  Traver  also  had  the  privilege  to  discuss  the  project 
concept  with  RADM  Ken  Slaght  (SPAWAR)  following  a  brief  at  NPS.  RADM  Slaght 
offered  several  useful  insights  specifically  concerning  the  direction  of  the  project  and  the 

potential  benefit  to  the  Navy.  In  addition,  several  sources  of  information  were  provided 
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by  SPAWAR  and  Northrop  Grumman  to  initiate  understanding  of  both  the  current  and 
alternative  NDEX  design  processes.  Finally,  one  teleconference  was  held  with  Mrs. 
Susan  Senese  (Co-Chairperson,  PEO-C4I  &  Space)  and  Mr.  Bob  Buckley  (SPAWAR 
04R3). 

The  NDEX  project  calls  for  a  change  in  the  way  C4I  equipment  is  configured  for 
installation  onboard  United  States  Navy  ships.  The  following  explanation  of  NDEX  is 
provided  by  Mr.  Dalton  in  a  working  paper  (White  Paper): 

NDEX  (Network  Design  Exchange)  is  a  series  of  products  and  processes 
designed  to  fully  integrate,  standardize,  automate,  control,  disseminate, 
and  manage  all  levels  of  design.  From  inception  (Development  of  ICD’s) 
to  installation  (SID’s),  NDEX  uses  a  combination  of  existing 
commercially  successful  programs,  a  web-based  document  management 
system,  and  mobile  CAD/Engineer  teams  to  significantly  improve  the 
entire  spectrum  of  design  products  and  processes.  (Dalton  1) 

NDEX  provides  the  ability  for  designers  to  input  the  characteristics  of  a  new  design 
directly  into  a  class  design  baseline  database  and  ensure  the  design  will  fit  in  the  ship’s 
configuration.  It  may  reveal  that  certain  ship  infrastructure  changes  need  to  be 
considered  prior  to  further  design  to  make  the  system  cost  effective.  NDEX  is  a  tool  to 
help  designers  plan  installations  while  potentially  avoiding  several  long  and  costly  steps 
in  the  current  process. 

NDEX  meets  the  design  requirements  imposed  upon  the  Navy  in  the  21st-century 
and  achieves  the  visions  set  for  the  21st-century  Navy.  These  requirements  and  the  vision 
NDEX  supports  are  covered  in  detail  in  Chapter  II. 


C.  RESEARCH  OBJECTIVES 

The  immediate  objective  of  this  effort  is  to  develop  a  workable  NROI  formula  for 
use  within  the  SPAWAR  organization,  specifically  for  the  NDEX  project.  However,  the 
underlying  objective  is  to  validate  an  approach  to  developing  NROI  formulae  for  use  in 
the  public  sector,  and  specifically  in  the  Department  of  Defense.  The  further 
development  and  application  of  the  JBG  NROI  formula  is  reported  in  this  report  to 
provide  the  reader  with  a  road  map  to  follow  in  future  applications. 
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D.  RESEARCH  METHODOLOGY 


At  the  end  of  2004,  Dr.  Jones  and  two  NPS  students,  LT  Joshua  Bigham  and  LT 
Thomas  Goudreau,  completed  another  Return  on  Investment  project  which  analyzed 
SPAWAR  PEO  C4I  Space  Command’s  return  on  new  ship  construction  (for  radio  control 
suite  rooms).  This  research  and  the  final  report  focused  on  prior  efforts  to  use  ROI  in  the 
public  sector  and  the  theoretical  development  of  the  JBG  NROI  formula,  stated  below 
(Bigham  33): 

PM*0. 1  +  ENG  *  0.4  +  SM  *  0. 15  +  PR  *0.4  +  INST  *  0.3  +  SS  *  0.2  +  MM  *0.1  +  Avoidance  Costs 

NROI  = - 

Radio  Control  Suite  Program  Costs 


As  stated  in  their  project,  this  formula  has  limitations  in  applicability,  specifically  it  is 
relevant  only  to  the  specific  platforms  for  which  it  was  developed,  but  the  methodology 
“of  defining  how  an  organization  adds  value  via  its  products  and  services”  (Bigham  55) 
provided  a  foundation  from  which  our  current  research  proceeded  from  without  retracing 
their  steps.  Therefore,  proceeding  from  this  methodological  foundation,  with  the  clear 
understanding  that  it  had  to  be  modified  and  supported  with  additional  theory  as 
necessary  during  our  progression,  we  have  arrived  at  an  ROI  calculation  for  NDEX. 
Chapter  III  details  the  evolution  of  the  calculation  and  its  final  derivation.  Chapter  IV 
details  the  application  of  the  methodology  to  the  management  decision  relating  to 
implementing  NDEX. 
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II.  CONTEXTUAL  BACKGROUND  CONSIDERATIONS  & 
REQUIREMENTS  AND  VISIONS  ACHIEVED  BY  NDEX 

A.  DEFINING  THE  NDEX  SYSTEM 

The  Network  Design  Exchange,  or  NDEX,  system  is  designed  to  change  the  way 
C4I  equipment  is  configured  for  installation  onboard  United  States  Navy  ships.  There  are 
several  known  components  in  this  system  and  in  the  design  process  that  in  the  inception 
stage  we  have  called  “NDEX.”  These  include  AutoCAD,  Web-based  collaboration 
software,  and  on-site  laptop  data  entry  processes. 

NDEX  is  based  on  commercially  available  software  that  will  allow  the  creation  of 
ship’s  drawings  and  designs  on  laptop  computers  using  Computer  Aided  Design  (CAD) 
applications  and  Class  Design  Baseline  (CDB)  drawings  that  show  how  a  ship  was 
configured  at  construction.  With  NDEX,  design  time  can  be  shortened  to  weeks  if  time 
criticality  is  necessary.  NDEX  allows  a  considerable  amount  of  time  to  be  saved  by 
eliminating  a  lot  of  the  middle-man  features.  For  example,  the  current  process  requires 
that  a  site  visit  is  conducted  and  sketches  are  made  for  equipment  layout.  The  sketches 
are  brought  back  to  an  office  where  they  are  given  to  an  individual  who  makes  drawings 
based  on  information  gathered  during  the  site  visit.  Often,  mistakes  are  made  based  on 
translation  issues.  With  NDEX,  a  laptop  computer  is  brought  to  the  site  visit  and 
drawings  are  made  during  the  visit  based  on  the  class  design  baseline  (CDB)  drawings. 
A  library  of  components,  known  as  Installation  Control  Drawings  (ICDs),  will  be 
maintained  and  available  to  click  and  paste  when  adding  equipment  that  is  in  the  space 
but  not  shown  on  the  CDB  drawings. 

NDEX  will  change  the  way  C4I  equipment  is  configured  for  installation  onboard 
ships.  The  following  explanation  of  NDEX  is  provided  by  Mr.  Dalton  in  a  working  paper 
(White  Paper): 

NDEX  (Network  Design  Exchange)  is  a  series  of  products  and  processes 
designed  to  fully  integrate,  standardize,  automate,  control,  disseminate, 
and  manage  all  levels  of  design.  From  inception  (Development  of  ICD’s) 
to  installation  (SID’s),  NDEX  uses  a  combination  of  existing 
commercially  successful  programs,  a  web-based  document  management 
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system,  and  mobile  CAD/Engineer  teams  to  significantly  improve  the 

entire  spectrum  of  design  products  and  processes.  (Dalton  1) 

NDEX  is  in  its  conceptual  stage,  with  component  systems  in  all  stages  of  development, 
from  being  unfunded  and  not  initiated  to  having  already  been  used  to  support  both  new 
installations  and  back-fits.  A  significant  potential  advantage  of  the  system  is  the 
avoidance  of  redundant  effort  and  inefficient  labor  practices  used  during  each  design 
process  step.  Time  and  labor  savings  provide  a  decent  return  on  investment,  but  NDEX 
also  yields  significant  improvements  related  to  mission  readiness  as  well  as  knowledge 
capture.  These  benefits  are  all  considered  as  a  part  of  the  return  on  investment  equation. 

NDEX  improves  design  time  to  a  matter  of  weeks  vice  months.  NDEX  saves  a 
considerable  amount  of  time  by  merging  the  data  gathering  and  drawing  steps  of  the 
current  process  into  a  single  process  step.  The  current  process  requires  that  a  site  visit  is 
conducted  and  the  engineer  conducting  the  visit  sketches  the  existing  configuration  and 
the  proposed  equipment  installation  layout.  The  sketches  are  brought  from  the  ship  back 
to  the  office  where  they  are  given  to  a  draftsman  (or  CAD  operator)  who  makes  drawings 
based  on  information  gathered  during  the  engineer’s  site  visit.  Often,  mistakes  are  made 
based  on  translation  issues  where  the  CAD  operator  may  misinterpret  the  intent  of  the 
engineer.  With  NDEX,  a  laptop  computer  is  brought  to  the  site  visit  by  the  engineer  and 
his  or  her  supporting  CAD  operator.  During  the  site  visit,  initial  drawings  or  changes  are 
made  to  establish  or  update  the  Class  Design  Baseline  (CDB)  drawings.  A  library  of 
Installation  Control  Drawings  (ICD’s)  contains  component  data  that  allows  for  a  point- 
and-click  pasting  for  automatic  integration;  or,  detailed  drawing  and  component  data  for 
efficient  facilitated  integration  of  component  additions  into  the  existing  system  being 
altered.  This  on-site  process  allows  for  highly  efficient  spatial  planning  for  the  new 
components  and  accounts  for  equipment  in  the  space  but  not  shown  on  the  existing  CDB 
drawings.  This  work,  which  creates  what  is  known  as  a  Ship  Change  Document  (SCD), 
can  be  transmitted  and  approved  while  still  on  the  site  visit  if  necessary  to  facilitate 
installations.  This  could  have  significant  benefits  to  ships  overseas,  especially  those 
subject  to  “Sea  Swap”  that  will  remain  overseas  for  significant  periods  of  time  and  be 
supported  by  rotating  crews. 
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Although  the  current  design  process  presently  remains  effective,  it  is  highly 
inefficient,  costly,  and  error-prone.  It  is  an  antiquated  system  that  is  labor  intensive  and 
uses  manual  drawings  to  design  compartment  utilization  and  ensure  the  proposed 
equipment  to  be  installed  will  fit  in  the  correct  space  allotted.  The  current  process  also 
takes  a  considerable  amount  of  time,  upwards  of  18  months  from  start  to  finish,  and 
requires  many  unnecessary  steps  to  accomplish  the  design.  These  steps  are  also  subject 
to  requiring  rework  to  achieve  any  level  of  accuracy.  Furthermore,  due  to  the  extended 
time  lag  with  the  current  process,  it  may  be  possible  for  projects  in  the  same  physical 
shipboard  compartment  to  overlap  and  have  the  same  equipment  or  ancillary  equipment, 
such  as  power  sources,  earmarked  for  more  than  one  installation.  Some  past  installations 
were  completed  without  SCD’s  completed  due  to  the  level  of  effort  required  and  the 
abundant  inefficiencies  in  the  process.  All  these  inefficiencies  are  captured  in  the  current 
budgeting  process  as  well,  increasing  the  overall  cost  of  design  funding  at  SPAWAR. 

In  addition,  the  current  process  is  very  slow  in  adapting  to  change.  During  the 
shipcheck  and  SCD  creation  process,  when  ICD’s  change  or  when  new  technology 
becomes  available  and  desired,  the  result  is  a  return  to  the  initial  steps  of  design.  Thus, 
the  process  starts  all  over  again.  This  leads  to  numerous  delays  and  a  great  amount  of 
retracing  steps  in  order  to  complete  the  installation.  In  the  meantime,  because  of  all  of 
the  delays,  it  is  likely  that  the  final  installation  will  be  antiquated  technologically  before 
the  end-users  even  begin  to  use  the  product.  This  directly  impacts  the  capabilities  in  the 
fleet. 

Another  feature  NDEX  provides  that  is  currently  unavailable  is  the  ability  for 
designers  to  input  the  characteristics  of  a  new  design  directly  in  to  the  class  design 
baseline  database  and  see  if  the  design  will  even  be  a  fit.  It  may  reveal  that  certain  ship 
infrastructure  changes  need  to  be  considered  prior  to  further  design  to  make  the  system 
cost  effective.  It  is  not  unheard  of  for  planners  to  find  that  their  new  equipment  will  not 
work  correctly  due  to  inadequate  electrical  power  sources  and  air  conditioning,  therefore 
requiring  further  changes.  With  this  in  mind,  major  changes  to  the  electrical  and  HVAC 
systems  will  need  to  be  perfonned  at  considerable  costs  that  were  yet  unidentified, 
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therefore  driving  the  install  cost  substantially  above  the  initial  planning  figures.  Simply 
stated,  this  tool  will  help  designers  plan  installations  while  potentially  savings 
tremendous  amounts  of  money 

While  the  impact  of  the  NDEX  design  process  will  also  impact  installation  costs 
and  rework,  we  have  limited  the  measurement  of  installation  effects  to  include  only 
design-related  installation  Change  Order  Request  Notifications  (CORNs).  In  addition, 
our  perspective  for  computing  benefits  is  from  that  of  SPAWAR,  so  our  measures  are  of 
SPAWAR’s  benefit,  not  that  of  other  stakeholders  (customers,  planning  yards,  etc.). 
However,  we  understand  that  NDEX  will  impact  these  stakeholders,  and  even  improve 
the  value  they  add  to  the  designs,  and  these  effects  are  measured.  So,  in  summary,  for  the 
purposes  of  our  research,  NDEX  encompasses  all  hardware,  software,  manpower,  and 
process  changes  in  the  design  system  that  impact  design  costing  and  design-related 
installation  CORNs.  Therefore,  we  attempt  to  measure  the  benefits  and  value  to 
SPAWAR  of  the  design  process  only,  not  the  installation  process. 

B.  2 1  st-CENTURY  REQUIREMENTS  AND  VISION  SUPPORTED  BY  NDEX 

Several  current  business  trends  and  resulting  initiatives  taking  place  in 
government,  the  Department  of  Defense,  and  the  Navy  impact  or  are  supported  by 
NDEX.  It  is  important  that  these  be  understood,  and  that  NDEX  be  considered  in  the 
context  of  all  the  changes  going  on  around  it  vice  considering  NDEX  in  a  vacuum.  Some 
of  these  include  ROI  measurement,  CNO  initiatives,  Fleet  Readiness  Plan  (FRP), 
FORCEnet,  Spiral  Acquisition,  and  Next-Generation  Manufacturing  Technology 
Initiative  (NGMTI),  to  name  a  few.  A  brief  initial  analysis  of  these  contextual  matters 
and  the  implications  of  these  to  NDEX  are  provided  below. 

First,  there  are  several  examples  of  the  Navy  being  criticized  for  its  inability  to 
accurately  assess  a  return  on  investment  related  to  its  programming  in  order  to  justify  its 
budget.  In  a  June  2003  report,  the  Government  Accountability  Office  (GAO)  stated, 
“Without  information  that  links  funding  to  readiness,  the  Navy's  budget  package  does  not 
provide  Congress  the  return  on  readiness  investment  information  it  may  need  to  make 
resource  decisions”  (GAO). 
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The  GAO  has  also  been  critical  of  the  evaluation  criteria  used  in  procurement,  as  the 
Navy  has  misapplied  ROI  formulas  to  business  decisions: 

Because  of  the  long-term  nature  of  these  investments,  they  typically  do  not 
yield  savings  in  the  early  years  while  initial  costs  are  being  incurred. 
According  to  the  Navy's  most  recent  assessment,  62  approved  aviation 
projects  yielded  about  $2  million  in  net  savings  from  fiscal  year  1997 
through  fiscal  year  2002.  These  projects,  along  with  1 1  forthcoming  ones, 
are  expected  to  generate  additional  savings  of  approximately  $785  million 
from  fiscal  year  2003  to  fiscal  year  2010. [Footnote  26]  In  addition,  Navy 
officials  noted  that  unmeasured  savings  may  accrue  through  cost 
avoidance  resulting  from  reduced  maintenance,  processing,  and 
transportation  of  broken  or  defective  items.  Navy  officials  told  us  that  the 
service  is  reviewing  plans  to  facilitate  project  approval  by  relaxing  current 
return  on  investment  criteria.  Management  attention  to  the  investment 
criteria  could  expand  the  number  of  eligible  parts,  help  mitigate  spare 
parts  shortages,  and  increase  the  readiness  return  on  investment.  (GAO) 


As  these  GAO  statements  show,  ROI  is  important  in  making  budget  claims,  but  can  be 
misleading  if  we  are  not  careful  to  ensure  analysis  over  the  life-cycle  of  the  project  in 
question.  These  are  just  two  of  many  ROI-related  issues  that  GAO  or  others  have  called 
into  question  concerning  the  DOD’s  and,  in  particular,  the  Navy’s  business  practices.  It 
is  evident  that  the  Navy  must  link  investments  to  outcomes  and  determine  the  value  of 
those  outcomes  in  order  to  quantify  ROI  and  present  justification  upon  request  at  any 
moment  throughout  the  program  life  cycle.  Therefore,  NDEX  must  be  considered  in 
light  of  this  framework,  and  analyzed  over  the  long-term  for  value.  The  resulting  ROI 
analysis  may  then  be  used  in  the  Planning,  Programming,  Budgeting,  and  Execution 
System  (PPBES)  if  necessary  in  order  to  justify  and  fully  fund  NDEX. 

Current  guidance  from  the  Office  of  Management  and  Budget  (OMB)  relating  to 
NDEX  that  supports  this  project  is  provided  in  CIRCULAR  NO.  A- 130,  Revised, 
(Transmittal  Memorandum  No.  4).  In  this  memorandum,  OMB  requests  that  agencies 
and  executive  departments  detennine  ROI  for  all  Federal  Information  Resources. 
Specifically,  the  memorandum  requests  that  agencies: 

(v)  Demonstrate  a  projected  return  on  the  investment  that  is  clearly  equal 
to  or  better  than  alternative  uses  of  available  public  resources.  The  return 
may  include  improved  mission  performance  in  accordance  with  GPRA 
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measures,  reduced  cost,  increased  quality,  speed,  or  flexibility;  as  well  as 
increased  customer  and  employee  satisfaction.  The  return  should  reflect 
such  risk  factors  as  the  project's  technical  complexity,  the  agency's 
management  capacity,  the  likelihood  of  cost  overruns,  and  the 
consequences  of  under-  or  non-performance.  Return  on  investment  should, 
where  appropriate,  reflect  actual  returns  observed  through  pilot  projects 
and  prototypes; 

(vi)  Prepare  and  update  a  benefit-cost  analysis  (BCA)  for  each  information 
system  throughout  its  life  cycle.  A  BCA  will  provide  a  level  of  detail 
proportionate  to  the  size  of  the  investment,  rely  on  systematic  measures  of 
mission  perfonnance,  and  be  consistent  with  the  methodology  described  in 
OMB  Circular  No.  A-94,  "Guidelines  and  Discount  Rates  for  Benefit-Cost 
Analysis  of  Federal  Programs;  (OMB) 


OMB’s  ROI  guidance  follows  the  President’s  lead  on  measuring  perfonnance,  both 
anticipated  performance  prior  to  investing  and  metrics  that  measure  performance  over  the 
life  of  an  agency  or  program.  In  addition,  MID-913  changes  to  the  PPBES  seek  to  focus 
efforts  more  on  execution.  Therefore,  our  effort  to  measure  ROI  for  NDEX  can  be 
viewed  as  an  effort  to  both  comply  with  current  guidance  and  to  further  develop  the 
Navy’s  ROI  measuring  process. 

Within  the  Navy,  the  fonner  Chief  of  Naval  Operations,  Admiral  Vernon  Clark,  had 
long  been  a  proponent  of  capturing  the  ROI  inherent  in  our  business  practices.  In  his 
latest  direction  to  the  Navy,  the  “2005  CNO  Guidance”,  ROI  considerations  were  a 
recurring  theme,  appearing  six  times: 

•  Ensure  savings  are  harvested  and  returned  to  the  leadership  for 
reallocation  against  other  Navy  priorities. 

•  We  established  the  Assistant  CNO  for  Infonnation  Technology  (ACNO- 
IT)  to  promote  Navy-wide  alignment  between  warfighting  and  business 
information  technologies,  and  to  ensure  IT  investments  and  resources  are 
targeted  for  highest  value  efforts  and  return  on  investment. 

•  Conduct  an  Expeditionary  Strike  Group  Sea  Swap  experiment  in  FY05. 
While  awaiting  live  testing,  conduct  computer  modeling  simulation  to 
provide  initial  assessments  of  feasibility  and  return  on  investment.  (CFFC 
by  Jun  05) 

•  Review  ship  (preventative/routine/intermediate/depot)  maintenance 
practices  and  quantify  corresponding  return  on  investment.  Report  on 
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innovative  ways  to  support  FRP  readiness  while  preserving  safe  and 
effective  operations.  (CFFC  lead,  NAVSEA,  TYCOMs,  by  May  05) 

•  [Develop]  an  analytical  process  for  warfighting  “wholeness”  and  a 
methodology  by  which  modernization  plans  can  be  evaluated  for  return 
on  investment  (e.g.,  platform  life  cycle  cost),  overall  value  to  warfighting 
and  risk.  (OPNAV  N6/N7) 

•  Facilitate  business  process  transfonnation  and  foster  a  culture  of 
productivity  and  continuous  improvement  enterprise- wide.  Develop  and 
advocate  high  potential,  cross-functional  enterprise  initiatives  and  ensure 
enhanced  performance  and  organizational  efficiencies  are  not  lost  in  the 
“white  space.”  Ensure  savings  are  harvested  and  returned  to  the 
corporation  for  reallocation  against  other  corporate  Navy  priorities. 
(Boldface  added  for  our  emphasis)  (Clark,  2005) 


NDEX  meets  the  spirit  of  all  of  the  above  criteria  as  outlined  by  the  CNO.  It  will  likely 
harvest  and  return  savings  to  SPAWAR  and  the  Navy  by  automating,  updating,  and 
streamlining  the  design  process.  In  addition,  it  specifically  supports  the  fourth  item,  to 
“review  ship  maintenance  practices  and  quantify  corresponding  return  on  investment” 
(Clark  2005).  However,  the  ability  to  greatly  improve  support  the  Fleet  Readiness  Plan 
(FRP)  may  be  the  greatest  advantage  of  NDEX. 

NDEX  achieves  the  CNO’s  goal  of  being  an  “innovative  way  to  support  FRP 
readiness  while  preserving  safe  and  effective  operations”  (Clark  2005).  This  is  a  very 
important  value-added  consideration,  as  the  current  design  process  is  apparently 
struggling  to  meet  the  challenges  presented  by  FRP.  FRP  shortens  the  turnaround 
process  for  ships,  requiring  compressed  maintenance  availability  and  training  periods  in 
order  to  ensure  the  fleet  maintains  a  higher  state  of  readiness.  Previously,  ships  would  go 
through  cycles  of  readiness  which  would  peak  on  deployment  or  just  prior  to  a 
deployment.  The  cycles  were  detrimental  to  readiness  and  created  periods  where  if  called 
upon,  ships  might  not  have  been  ready  for  an  assigned  mission.  FRP  has  attempted  to 
eliminate  the  cyclical  trends  and  focus  on  a  steady  state  of  readiness.  NDEX  will  mesh 
into  the  goal  of  FRP  by  providing  shortened  installation  times  and  overall  greater  fleet 
readiness. 

NDEX  will  also  possibly  lead  to  savings  in  the  platfonn  life  cycle  costs,  returning 


value  to  the  warfighter.  The  CNO’s  interest  items  should  continue  to  be  monitored  and 
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NDEX  “fit”  into  the  overall  objectives  of  the  Navy.  As  summarized  above,  this  “fit” 
should  be  natural  and  advantageous  in  selling  the  advantages  of  NDEX.  How  these 
advantages  will  impact  our  ROI  calculation  should  also  be  considered  vigorously  as  we 
proceed. 

As  might  be  expected,  NDEX  supports  SPA WAR’s  “Enterprise  Strategic  Plan 
2005-2010.”  The  plan’s  “Framework  for  Information  Systems”  challenges  the 
organization  to  leverage  cost  savings  toward  FORCEnet,  or  toward  the  warfighting 
capability  in  the  fleet.  “We  must  leverage  business  infonnation  technology  investments, 
strategies,  and  efficiencies  to  reduce  costs  associated  with  FORCEnet  delivery  and 
deployment  while  ensuring  alignment  with  business  information  technology  governance 
policies  and  procedures  of  ACNO  IT  and  PEO  IT.”  (SPAWAR  12)  While  improving 
delivery,  NDEX  leverages  already  established  technologies  to  reduce  costs  inherent  in  the 
current  process.  Achieving  this  change  in  process  will  contribute  significantly  to 
SPAWAR  meeting  its  strategic  goal  over  the  coming  years. 

In  recent  years,  Spiral  Acquisition  methods  have  been  considered  a  key 
component  of  future  acquisition  plans.  In  order  to  support  spiral  acquisition,  in  which 
new  technologies  continue  to  be  delivered  as  they  are  developed,  close  coordination  and 
readily  available  technical  infonnation  are  critical.  NDEX  provides  this  coordination  and 
ready  infonnation.  The  Program  Executive  Officer/  Systems  Command 
(PEO/SYSCOM)  Commanders’  Conference  is  “one  of  the  longest  running  forums  to 
review  progress  in  achieving  DoD’s  acquisition  refonn  objectives.  Nearly  500 
representatives  of  the  DoD  acquisition  community  ...  exchange  success  stories  and 
lessons  learned”  (Reed  10).  At  the  ninth  conference  in  this  continuing  series,  Dr.  Jacques 
S.  Gansler,  Under  Secretary  of  Defense  (Acquisition,  Technology,  and  Logistics),  noted 
that  changes  in  technology  and  the  threat  environment  dictate  dramatic  changes  in  the 
way  DoD  and  industry  operate:  “We  need  to  change  the  way  we  buy  and  field  new 
systems.  We  need  to  follow  a  spiral  requirements  process,  where  we  put  things  in  the 
field  and  then  improve  them.  In  addition,  unless  we  get  ways  to  dramatically  reduce  the 
costs  of  these  systems,  we  won’t  be  able  to  afford  them”  (Reed  1 1).  If  the  advantage  of 
spiral  acquisition  is  to  be  realized,  the  supporting  design  system  must  support  quick 
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decisions  and  delivery  of  “plug  and  play”  components  in  the  mission  architecture.  In 
order  to  facilitate  this,  NDEX’s  database  will  provide  the  information  necessary  for  spiral 
acquisition,  and  its  interface  will  allow  coordination  and  efficiency  in  the  process  of 
spiral  delivery  of  new  technology. 

Another  initiative  which  NDEX  will  support  is  SHIPMAIN.  SHIPMAIN  seeks  to 
provide  “a  common  planning  process  for  surface  ship  maintenance  and  alterations”  and 
“increase  the  efficiency  of  the  process  without  compromising  its  effectiveness” 
(SHIPMAIN  3).  SHIPMAIN  integrates  the  ship’s  Commanding  Officer,  the  Port 
Engineer,  and  the  SUPSHIP  Manager  at  the  shipyard  in  a  maintenance  team  that 
coordinates  the  work  required  at  a  Planning  Board  for  Maintenance.  One  goal  is  to 
complete  installations  and  work  whenever  possible,  vice  waiting  for  a  lengthier 
availability  period.  In  this  effort,  NDEX  will  provide  the  backbone  for  design  support 
that  can  be  leveraged  toward  better  definition  of  the  C4I  task  to  ensure  it  can  be 
completed  on-schedule  to  support  the  ship’s  schedule  and  operations.  SHIPMAIN,  in  its 
support  for  the  FRP,  also  seeks  to  ensure  surge  capability.  Again,  NDEX  will  be  more 
responsive  and  will  better  handle  this  need  than  the  current  process. 

Two  lesser  known  initiatives  should  be  kept  in  mind  while  NDEX  is  developed. 
In  fact,  one  such  DOD  initiative,  the  Next-Generation  Manufacturing  Technology 
Initiative  (NGMTI)  may  be  a  source  of  support  for  NDEX.  NGMTI’s  objective  is  to: 

...develop  a  national  manufacturing  technology  investment  strategy  to 
accelerate  the  transformation  of  the  U.S.  industrial  base.  NGMTI  seeks  to 
energize  a  national  consensus  for  investing  in  high-  leverage,  high-  impact 
manufacturing  technologies  that  enable  faster  delivery  of  affordable 
systems  for  defense  while  at  the  same  time  improving  the  global 
competitiveness  of  U.S.  manufacturers.  (Graves  1) 

NDEX,  in  enabling  faster  delivery,  may  support  the  objectives  of  NGMTI  and  thus 

qualify  as  of  interest.  NGMTI  General  Manager  Gerry  Graves  of  the  Advanced 

Technology  Institute  (ATI)  stated,  "Our  objective  is  to  radically  enhance  the  nation's 

return  on  its  manufacturing  technology  investments.  We  want  to  reestablish  U.S. 

leadership  in  manufacturing  science  and  technology  by  delivering  a  plan  to  double  the 

nation's  manufacturing  technology  investments  and  increase  the  return  on  those 

investments  by  a  factor  of  ten"  (Graves  1).  NDEX,  while  not  in  the  private  sector,  is 
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interesting  when  related  to  NGMTI,  because  it  offers  DOD  an  internal  improvement  that 
may  deliver  the  same  results  sought  by  NGMTI.  "For  the  defense  community,  this  means 
shortening  the  time  and  cost  to  move  new  weapons  from  concept  to  delivery"  (Graves  1), 
said  Richard  Neal  of  IMTI,  the  NGMTI  Technical  Director.  NDEX  is  SPAWAR’s  tool 
which  will  improve  this  delivery  process  for  C4I  systems. 

Worth  noting  as  it  relates  to  NDEX  is  the  Information  Lifecycle  Management 
(ILM)  system.  ILM  is  a  “strategy  that  uses  people,  processes  and  technology  to  store  and 
tap  critical  agency  data  throughout  its  lifespan  of  value.  For  many  agencies,  smart  use  of 
information  has  become  a  differentiator,  particularly  as  technology  provides  information 
access  within  and  among  agencies  who  need  to  collaborate”  (Infonnation  Lifecycle 
Management  1).  NDEX  may  be  seen  as  similar  to  an  ILM  system,  and  thus  more 
research  should  be  done  on  ILM.  In  addition,  ILM  addresses  several  concerns  in  the 
current  process,  including  rising  costs  of  technology,  increased  labor  to  support  back-fits, 
and  aging  technical  experts  that  will  soon  be  retiring,  leaving  a  void  in  corporate 
knowledge.  If  ILM  can  deliver  as  advertised,  it  is  an  important  source  of  comparison 
with  NDEX:  “ILM  is  a  strategy  that  provides  a  return  on  investment  that  frees  resources 
to  sustain  operations  with  fewer  resources.  ILM  is  being  discovered  as  a  unique  weapon 
against  flat  information  technology  budgets,  increasing  labor  costs  and  an  aging 
workforce”  (Information  Lifecycle  Management  2).  The  same  could  be  said  about 
NDEX. 


C.  ANALOGOUS  IMPLEMENTATIONS  OF  CAD  MANAGEMENT 
SYSTEMS 

In  a  Congressional  Research  Service  (CRS)  updated  report  to  Congress  on  June 

23,  2005,  entitled  “Navy  Ship  Acquisition:  Options  for  Lower-Cost  Ship  Designs — Issues 

for  Congress,”  the  Government  Accounting  Office’s  (GAO)  February  28,2005  report  is 

cited,  in  which  numerous  problems  in  the  Navy’s  ship  cost  estimating  process  were 

noted.  Of  note,  one  conclusion  drawn  was  that  “contract  prices  were  negotiated  and 

budgets  established  without  making  full  use  of  design  knowledge  and  construction 

experience”  (O’Rourke,  CRS-31).  In  addition,  the  CRS  report  recommended  one  option 

for  reducing  design  costs  is  to  “improve  the  operating  efficiency  of  yards  building  Navy 
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ships  by  incorporating  more  advanced  design  and  production  processes  and  equipment” 
(O’Rourke,  CRS-34).  In  light  of  this  pressing  current  need,  NDEX  delivers  an  advanced 
alternative  for  C4I  configuration  design  process  reengineering.  However,  SPAWAR  is 
not  alone  in  its  attempt  to  transform  the  way  in  which  design  work  is  performed. 

Several  private  and  public  industry  initiatives  parallel  the  NDEX  program.  Most 
are  defined  as  collaborative  engineering,  online  collaboration,  or  product  lifecycle 
management  initiatives.  A  few  of  these  initiatives  will  be  covered  here  for  the  point  of 
comparison  with  NDEX. 

In  China,  Wuchang  Shipyard  announced  in  June  of  2005  that  it  had  implemented 
Product  Development  Company’s  Shipbuilding  Solution  which  comprises  several 
software,  hardware,  and  process  innovations  “which  will  be  used  as  a  production  design 
management  platform...  to  bring  about  revolutionary  changes  in  Wuchang  Shipyard’s 
business  processes,  particularly  in  the  way  product  data  is  managed” 
(www.businesswire.com).  Wuchang  Shipyard  was  founded  in  1934  and  is  one  of 
China’s  largest  shipyards,  with  the  most  modern  design,  build,  and  repair  facilities  in 
China  for  civil,  military,  and  tourist  ships  used  in  China  and  exported  around  the  world. 
Wuchang  Shipyard  expects  that  this  “collaborative  tool  for  the  R&D,  manufacturing  and 
production  departments,  [will  enable]  concurrent  engineering,  shortening  product  design 
lead-time  and  accelerating  time-to-market”  (www.businesswire.com).  The  benefits  of 
this  initiative  are  not  yet  known. 

Design  collaboration,  while  challenging,  provides  a  significant  return.  Gartner 
research  director,  Mark  Halpern,  estimates  that  “design-collaboration  tools  help 
companies  cut  the  time  it  takes  to  finish  a  design,  typically  by  about  30%. ..  that’s  very 
significant  in  a  marketplace  where  getting  a  new  product  to  market  before  the 
competition  is  a  key  competitive  advantage”  (Gonsalves  2).  In  the  DOD,  this 
competitive  advantage  has  been  a  mantra  for  years,  and  design  collaboration  will  ensure 
that  our  advantage  of  being  on  the  leading  edge  of  defense  technology  can  be  maintained, 
or  even  accelerated. 
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Several  U.S.  and  worldwide  companies  have  reported  the  results  of  establishing 
online  collaboration  tools.  DaimlerChrysler  reported  that  “Compared  with  the  old  means 
of  couriers,  faxes,  and  face-to-face  design  meetings,  [it]  has  cut  60%  to  90%  of  the  time  it 
takes  to  communicate  design  changes  to  a  supplier  and  get  required  changes  back” 
(Gonsalves  2).  Karenann  Terrell,  director  of  its  E-business  operation,  E-connect  stated, 
“Any  change  to  a  design  is  immediately  communicated  to  everyone,  and  the  impact  on 
their  piece  of  the  design  is  known  right  away”  (Gonsalves  2).  Similarly,  Lockheed 
Martin  used  “proprietary  collaborative  design  technology,  a  blend  of  web-based  tools  that 
use  enterprise  data  management  software  and  simulation  technology  to  reduce  design 
cycles  by  an  astounding  40%  to  80%”  (Krouse  1).  In  addition,  Babcock  Power  in 
Worcester,  Massachusetts,  a  maker  of  “scrubbing”  systems  for  power  plants  realized  a 
90%  reduction  in  search  time  for  project  infonnation  (Krouse  5).  How  did  Babcock 
Power  accomplish  this?  “Communication  about  drawings  and  other  documents  is  now 
done  online”  (Krouse  5).  Finally,  Gamesa,  a  Spanish  aircraft  and  helicopter  assemblies 
manufacturer  used  “EDS  Solutions  Teamcenter  to  reduce  design  errors  by  a  whopping 
88%”  (Krouse  5). 

One  effort  to  compute  ROI  as  it  relates  to  CAD  software  is  of  interest.  In  a  recent 
White  Paper,  AutoCAD  2000  was  evaluated  to  determine  the  performance  and 
productivity  ROI  of  upgrading  from  a  previous  version  of  CAD  software,  R14. 
Measured  from  the  perspective  of  a  sole  user  as  the  stakeholder,  the  ROI  was  measured  at 
347%  (www.cadresource.com/r2000).  While  not  completely  analogous,  it  is  expected 
that  NDEX  can  achieve  similar  efficiency  and  productivity  enhancements  for  those 
responsible  for  creating  and  maintaining  Navy  ships’  drawings. 

In  summary,  the  use  of  online  collaborative  design  tools  has  yielded  the  reported 
effects  summarized  in  Table  1. 
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Table  1.  Combined  Effects  of  CAD  and  Collaboration  Technology  Improvement 

Initiatives  In  Private  Industry 


Design  CAD/Collaborative  Technologies  Combined  Effects 

" 1 . 

30%  overall  reduction  in  time  to  finish  design 

"2. 

60%  to  90%  reduction  in  time  to  communicate  design  changes  to  supplier  and  get  back 

’S. 

40%  to  80%  reduction  in  design  cycle  time 

A 

90%  reduction  in  search  time  for  project  information 

lB. 

88%  reduction  in  design  errors 

r6. 

347%  ROI  on  sole  CAD  user's  performance  &  productivity 

These  effects  must  be  viewed  as  potential  results  of  not  only  implementing  the  new 
technology,  but  also  reengineering  the  processes  that  this  technology  will  support  in  order 
to  harvest  these  benefits.  NDEX  may  achieve  results  similar  to  many  of  these. 

In  addition  to  these  initiatives,  Table  2,  compiled  by  Dr.  Peter  Capell  of  the 
Carnegie  Mellon  Software  Engineering  Institute,  outlines  the  type  of  results  that  have 
been  achieved  by  process  improvement  efforts  related  to  acquisition,  software,  and 
design.  Dr.  Capell  cites  three  recurring  themes.  First,  he  states  that  process  improvement 
aims  at  “eliminating  errors  in  the  process  upstream.”  (Capell,  v)  NDEX  certainly  aims  to 
achieve  this  goal,  eliminating  costly  Change  Order  Request  Notifications  (CORNs)  and 
rework  in  the  design  process.  One  case  study  in  this  body  of  research  found  that  30%  of 
errors  found  at  integration  were  eliminated. 

The  second  recurring  theme  in  Dr.  CapelTs  research  is  that  “while  improvement 
efforts  are  typically  driven  by  cost  and  measured  by  return  on  investment,  quality 
attributes. . .  are  often  of  more  value  to  the  success  of  the  overall  project.”  (Capell,  v)  The 
value  of  these  additional  quality  attributes  is  difficult  to  quantitatively  measure. 
However,  some  of  these  quality  attributes  are  explained  in  the  Abrams  project,  for 
example,  capturing  the  “knowledge  and  expertise  of  current  PM  Abrams  personnel  for 
future  employees”  (Capell,  5).  This  same  quality  attribute  holds  true  for  NDEX.  By 
capturing  data,  future  design  professionals  will  have  a  tool  that  not  only  enables  them  to 
perform  more  quickly,  but  also  refer  to  existing  drawings  quickly  when  needed  to 
enhance  their  understanding  of  an  existing  problem. 
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The  third  recurring  theme  presented  by  Dr.  Capell  is  that  “most  improvement 
efforts  will  yield  benefits  as  long  as  those  efforts  follow  general  rules  and  use  appropriate 
methods.”  (Capell,  v)  In  implementing  NDEX,  establishing  these  rules  and  methods  for 
collaboration  and  design  will  lead  to  beneficial  results,  leading  to  the  question —  how 
much?  It  is  important  that  consideration  be  given  to  new  paradigms  relating  to  the 
acquisition  and  generation  of  designs  as  well  as  the  provision  of  design  support  services 
that  capture  the  full  benefits  of  working  in  a  collaborative  environment. 
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Table  2.  Program  benefits  related  to  Acquisition,  Design,  and  Software  Improvements 

(Capell  5-6) 


Organization 

Result 

Abrams  Project 
Manager  (PM) 

•  Empowered  the  PM  Abrams  organization  and  facilitated  M1A2  SEP  development 

•  "Leveled  the  playing”  field  with  the  contractor  by  providing  a  common  language  and  set 

of  standards 

•  Captured  the  knowledge  and  expertise  of  current  PM  Abrams  personnel  for  future  em¬ 
ployees 

•  Paved  way  for  transitioning  the  PM  Abrams  to  support  organizations  through  docu¬ 
mented  policies,  procedures,  and  practices 

Accenture 

Reported  5:1  benefit  as  a  ratio  of  hours  spent  on  Capability  Maturity  Model  Integration 
(CMMI)  process  activities 

Air  Force 
Institute  of 
Technology 

Cost  Performance  Index  (CPI):  The  multiple  comparison  test  showed  significant  differences 
between  CMM  Levels  1  and  2  and  between  CMM  Levels  1  and  3.  As  the  organization  in¬ 
creased  in  CMM  maturity,  its  CPI  general  approaches  1.00  with  generally  decreasing  varia¬ 
tion. 

Schedule  Performance  Index  (SPI):  At  all  CMM  rating  levels,  the  SPI  remains  close  to  1.00. 
However  for  projects  less  than  80%  complete,  the  performance  of  Level  1  organizations  was 
consistently  below  an  SPI  of  1.00.  As  the  institute  exceeded  the  80%  completed  milestone,  it 
increased  its  SPI  to  meet  the  required  1.00 

Boeing 

Australia 

•  Decreased  cost  to  correct  a  defect  (measured  over  18  months)  33% 

•  Decreased  time  to  deliver  releases  50  % 

•  Reduced  preparation,  conduct,  and  rework  from  pre-test  and  post-test  audits  60%  (result¬ 
ing  in  audits  passed  with  few  or  no  outstanding  actions). 

General  Motors 

•  Increased  the  percentage  of  milestones  met  from  approximately  50%  to  approximately 

95% 

•  Decreased  the  average  of  milestones  days  late  from  approximately  50  to  fewer  than  10 

Hughes  Aircraft 

Saved  $2  million  dollars  annually 

Other  improvements  include  certain  quality  of  life  attributes  such  as: 

•  Fewer  overtime  hours 

•  Lower  employee  turnover 

•  Better  organizational  image 

Jet  Propulsion 
Laboratory 

Reduced  defect  identification  and  correction  time  from  as  much  as  17  hours  to  less  than  2 

hours 

JP  Morgan  - 
Chase 

•  Reduced  post-release  defects 

•  Reduced  severity  of  post-release  defects 

•  Improved  predictability  of  scheduled  deliveries 

Litton  Data 
Systems 

Reduced  the  number  of  errors  found  during  systems  integration  30% 
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Lockheed 

Martin  MD  &  S 

•  Reduced  overhead  rate  4.5% 

•  Increased  software  productivity  30% 

Motorola 

Government 

Electronics 

Division 

Achieved  an  ROI  of  6.77  to  1  for  a  typical  1 00,000  Software  Lines  of  Code  (SLOC)  project. 
Such  a  project  would  span  18  months  and  require  20  software  engineers.  In  dollar  amounts, 
Motorola  saved  $61 1,200  after  investing  $90,180  in  a  typical  project. 

NASA  Goddard 

•  Reduced  cost  of  software  development  by  55% 

•  Decreased  cycle  time  by  40% 

•  Reduced  post-release  defect  rate  by  75% 

Northrop 

Grumman 

Electronics 

Systems 

Moving  from  Level  3  to  Level  4  resulted  in  20%  annual  gains  in  productivity  for  the  5-year 
reporting  period.  This  represented  a  10%  gain  over  their  previous  annual  improvement 
achievement.  The  acceleration  resulted  in  a  $25  million  annual  cost  avoidance  over  the  5-year 

time  span. 

Northrop 
Grumman  IT 

•  Achieved  13:1  ROI  calculated  as  defects  avoided  per  hour  spent  in  training  and  defect 

prevention 

•  Met  milestones  25  times  in  a  row  (with  high  quality  and  customer  satisfaction) 

•  Increased  focus  on  quality  over  “firefighting” 

•  Earned  rating  of  “exceptional”  in  every  applicable  category  on  its  Contractor  Perform¬ 
ance  Evaluation  Survey 

Oklahoma  City 
Air  Logistics 
Center 

•  Saved  $  1 1 .3  million  over  8  years 

•  Reduced  defects  90%  from  baseline  project  to  second  project 

•  Reduced  average  cost  of  a  TPS  maintenance  action  20%  over  the  last  two  years 

•  Achieved  10X  increase  in  productivity  based  on  source  code  produced  from  baseline  to 
most  recent  project  due  to  technology  as  well  as  CMM  improvements 

Raytheon 

Northeast 

Software 

Engineering 

Center 

•  Reduced  Cost-Performance  Index  variability  by  36% 

•  Reduced  Schedule  Performance  Index  variability  by  70% 

•  Reported  productivity  gains  in  the  following  percentages: 

-  1997-1999: 30% 

-  2000: 9% 

-  2001:11% 

-  2002: 6% 

Sanchez 
Computer 
Associates,  Inc. 

•  Saved  $2  million  in  first  6  months  due  to  early  detection  and  removal  of  defects 

•  Created  a  robust  training  program 

•  Applied  process  improvement  activities  to  functions  other  than  programming 

Schlumberger 

•  Reduced  validation  cycles  54% 

•  Improved  productivity  100% 

•  Increased  on-time  delivery  of  software  from  5 1  %  to  94% 

•  Reduced  post-release  defects  from  25%  to  10% 

Thales  TT&S 

Decreased  cost  and  schedule  variances  as  process  maturity  levels  increased 
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We  can  deduce  from  this  table  that  the  process  improvements  achieved  had 
significant  effects  within  the  organization,  saving  time  and  money.  Numbers  ranging 
from  10%  to  over  1300%  are  reported  for  the  various  “returns”  of  these  efforts.  In 
addition,  many  of  these  process  improvements  were  not  expensive  to  implement  in 
comparison  to  the  net  effect.  Finally,  as  stated  before,  not  all  effects  were  monetized,  but 
each  was  valuable  to  the  organization  and  improved  the  process  for  both  the 
organization’s  employees  and  customers.  Even  if  NDEX  were  to  fall  short  of  realizing 
the  significant  cost  avoidances  expected,  it  would  likely  prove  its  value  many  times  over 
to  both  the  customer  and  the  employees  currently  tasked  in  design  processes  by 
improving  the  efficiency  and  ease  of  managing  and  communicating  design  data. 
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III.  DEVELOPMENT  OF  THE  METHODOLOGY 


A.  OVERVIEW  OF  THE  JBG  NROI  MODEL 

As  stated,  the  JBG  NROI  formula  was  the  starting  point  for  arriving  at  a  measure 
of  ROI  for  NDEX.  The  original  formula  is  restated  below. 

PM*0. 1  +  ENG  *  0.4  +  SM  *  0. 1 5  +  PR  *0.4  +  INST  *0.3  + SS  *0.2  +  MM  *0.1  +  Avoidance  Costs 

NROI  = - 

Radio  Control  Suite  Program  Costs 


The  need  for  businesses  to  capture  return  on  investment  (ROI)  has  been  an 
industry  standard  for  many  years  and  is  one  metric  many  businesses  use  to  gauge 
financial  performance.  It  is  a  very  challenging  task  to  compute  ROI  for  the  public  sector 
since  the  normally  applied  fonnula  numerator  of  profit  or  net  operating  income  is 
difficult  to  capture  in  DoD  applications. 

There  have  been  numerous  attempts  to  calculate  ROI  in  the  public  sector  but  all 
have  met  mixed  results.  The  prior  Chief  of  Naval  Operations,  ADM  Vernon  Clark  placed 
much  emphasis  on  capturing  ROI  in  November  of  2003  in  an  Echelon  II  visit  feedback 
memo: 


This  is  an  area  where  we  are  struggling  the  most.  We  need  to  know  that 
we  are  making  the  right  type  and  level  of  investment.  We  have  made 
some  progress  in  understanding  what  we  are  investing  in  and  have  even 
made  progress  in  understanding  the  output  of  our  processes.  However,  we 
are  struggling  to  link  the  two  (investment  to  output).  We  need  to  model 
how  increases  or  decreases  in  investments  (people,  dollars,  and 
technology)  will  change  the  output.  (Clark,  2003) 

Traditional  ROI  computation  cannot  be  used  in  the  public  sector  as  freely  as  private 
enterprise.  The  reasoning  is  that  public  sector  organizations  do  not  produce  profits  or 
generate  revenues  as  their  outputs,  as  do  private  organizations.  It  is  difficult  to  measure 
how  much  benefit  or  value  is  derived  from  producing  a  new  ship  class  or  aircraft  type. 
While  vitally  important  to  national  security,  it  is  hard  to  detennine  how  much  return  is 
generated  on  that  investment. 
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There  have  been  many  attempts  to  quantify  ROI  in  the  public  sector  both  by 
international  governments  as  well  as  domestic  organizations.  The  Australian  government 
attempted  to  quantify  ROI  on  government  purchases  in  an  attempt  to  show  its  citizens 
that  it  is  making  proper  use  of  funds.  The  effort  resulted  in  criticism:  “There  is  a 
widespread  dissatisfaction  with  Defence’s  Performance  [regarding  the  Australian 
Defence. .  .use  of  funds]. .  .In  essence  we  have  a  credibility  problem”  (Bigham  16). 

The  Royal  New  Zealand  Navy  initiated  ROI  measures  on  a  program  used  to 
control  attrition  of  their  marine  engineers  by  providing  better  pay  and  bonuses. 
Essentially,  the  ROI  study  was  initiated  to  validate  the  cost  of  the  program  of  increasing 
the  compensation  of  the  marine  engineers  in  comparison  to  retention  rates  after  the 
program’s  implementation  (Bigham  16). 

The  United  States  Postal  Service  (USPS)  attempted  to  measure  ROI  by  using  an 
Economic  Value  Added  (EVA)  approach  which  measured  the  efficient  use  of  assets.  The 
EVA  effort  was  discarded  in  2002.  Of  note  is  that  the  USPS  is  one  of  the  few  public 
organizations  that  realize  a  profit  and  can  use  traditional  ROI  computations  in 
determining  financial  perfonnance  (Bigham  20).  In  addition,  the  Navy  Dental  Corps 
used  several  iterations  of  an  ROI  formula  in  an  attempt  to  quantify  their  perfonnance 
output  with  mixed  success  (Bigham  22). 

The  basis  for  our  formula  is  patterned  after  the  Jones-Bigham-Goudreau  notional 
ROI  formula  (JBG  NROI)  derived  in  an  attempt  to  measure  ROI  of  Radio  Control  Suites 
(RCS)  on  the  new  T-AKE  class  of  ship  (Bigham  33).  The  JBG  NROI  formula  was 
derived  over  a  period  of  10  months  and  had  a  total  of  four  iterations.  Early  iterations 
attempted  to  capture  weighted  vital  components  of  the  program.  It  wasn’t  until  the  third 
iteration  that  the  numerator  of  the  final  product  eventually  evolved  into  what  was  tenned 
Work  Breakdown  Structure  (WBS)  categories  and  applied  weights  to  each  category.  The 
fourth  and  final  iteration  fine-tuned  the  numerator  of  the  fonnula  to  compensate  for 
rework  costs  and  to  model  the  formula  after  a  traditional  ROI  formula. 

The  seven  WBS  categories  that  were  settled  upon  were  Program  Management, 
Engineering,  Supportability  Management,  Production  (integration),  Installation,  Shipyard 
Support,  and  Material  Management. 
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Weights,  or  value-added  factors  (VAFs),  of  each  WBS  category  were  settled  upon 
after  consulting  with  several  Program  managers  and  RCS  experts  from  both  the  T-AKE 
and  other  classes  of  ships  and  the  final  product  was  delivered.  An  example  of  the 
weighting  settled  upon  is  that  for  every  dollar  spent  in  Program  Management,  $1.10  is 
returned. 

B.  ADAPTATIONS  TO  THE  JBG  NROI  MODEL 

In  conversations  held  with  Mr.  Ed  Dalton  and  Mr.  Jay  Johnson,  we  agreed  that 
NDEX  added  value  to  the  existing  processes  in  addition  to  avoiding  costs.  First,  we 
believed  that  knowledge  capture  was  a  significant  benefit  derived  from  NDEX  that  was 
not  in  the  original  fonnula,  and  that  this  knowledge  would  be  leveraged  toward  the 
existing  process  to  deliver  better  design  support  in  the  ship  installation  and  alteration 
processes.  Second,  we  felt  that  customer  support  and  responsiveness  was  greatly 
improved  and  added  value  to  the  process.  Finally,  we  saw  acquisition  alternatives  would 
increase  and  be  more  accurate  due  to  the  infonnation  available  in  NDEX.  Much  later  in 
the  process,  we  realized  the  current  process  delivers  the  same  benefits,  but  that  NDEX  as 
a  tool  leverages  these  efforts  and  improves  the  resulting  impact  of  current  efforts. 

After  several  struggles  with  application,  we  realized  that  investment  effects  from 
investing  in  NDEX  did  not  behave  in  the  same  manner  as  direct  investment  in  a  program, 
such  as  the  Radio  Control  Suite  Program  originally  measured  with  the  JBG  NROI 
formula.  The  investment  in  NDEX  is  leveraged  toward  all  programs  supported  by 
designs.  Thus,  the  investment  in  NDEX  indirectly  results  in  improved  existing  processes 
in  program  management  and  processes  leading  to  installation  and  operation  of  the  system 
supported. 

In  returning  to  analyze  the  original  JBG  NROI  variables,  supporting  the  Work 
Breakdown  Structure  (WBS)  utilized  in  the  installation  process  was  important  for  both 
consistency  of  application  and  ease  of  tracking.  However,  how  to  account  for  the  benefit 
a  tool  like  NDEX  provided  to  the  existing  WBS  expense  model  was  not  directly  evident. 
We  determined  that  what  NDEX  does  as  a  tool  is  provide  better  information  and  design 
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data  throughout  the  process,  benefiting  other  work  on  a  scale  related  to  the  funding 
flowing  into  NDEX.  For  example,  funding  NDEX  at  a  $10M  value  to  employ  enough 
CAD  operators  to  complete  CDBs  for  over  100  ships  in  one  year  would  yield  a  benefit  to 
the  entire  WBS  process(es)  on  scale  with  the  $10M  invested  into  NDEX.  We  then 
determined  what  value  NDEX  would  provide  and  tied  it  to  NDEX  funding.  It  is 
important  to  note  that  in  terms  of  notional  benefit  the  benefit  may  not  be  seen  in  the 
bottom  line  of  these  processes.  However,  as  detailed  above,  the  benefits  of  NDEX  are 
not  completely  measurable.  For  example,  what  is  the  value  of  time-to-market  in  a 
warfare  environment?  The  answer  must  be  dependent  on  the  scenario,  however  the 
benefit  of  a  time-to-market  (to  install  and  operate  in  this  case)  advantage  surely  exists. 
These  benefits  at  a  lower  magnitude  are  provided  inherently  in  the  current  WBS 
processes.  The  value-added  by  NDEX  is  thus  the  multiplied  effect  it  has  on  the  WBS 
value  to  the  organization.  Even  if  this  benefit  is  realized  by  the  end-user,  it  results  in 
goodwill  benefit  for  the  organization  (in  this  case,  SPAWAR).  Thus,  we  captured 
through  a  NDEX  “Invested  Multiplier”  the  benefit  that  funding  NDEX  will  create  in  the 
ship  design  and  installation  process  as  reflected  in  WBS  format. 

We  were  left  with  choosing  to  apply  NDEX  to  all  programs  supported,  a  daunting 
task,  or  to  measure  the  notional  benefits  derived  by  NDEX  as  a  whole.  In  the  interest  of 
time,  we  chose  to  factor  the  “Invested  Multiplier”  to  measure  the  expected  whole. 
Application  and  measurement  of  the  formula  as  it  relates  to  an  individual  project  or 
program  would  be  of  interest  in  follow-on  research  and  would  perhaps  refine  or  verily  the 
“Invested  Multiplier”  factors. 

The  benefits  of  NDEX  will  likely  be  realized  and  increase  with  acceptance  and 
growth  of  the  system;  however,  an  implementation  period  is  expected  in  which  benefits 
are  not  realized  immediately.  A  “learning  curve”  effect  will  likely  be  encountered  while 
users  and  customers  adapt  to  the  capabilities  of  the  new 
system.  Therefore,  we  conservatively  lagged  benefit  one  year  behind  funding  as 
immediate  benefit  is  not  likely,  and  then  increased  benefits  by  adjusting  the  Invested 
Multiplier  up  over  time. 
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c. 


FINAL  NOTIONAL  ROI  FORMULA 


The  resulting  revised  JBG  NROI  formula  is  presented  below  as  it  relates  to  a 
program  that  acts  as  a  tool  that  can  be  leveraged  in  support  of  other  programs: 


JBG  NROI 


PMIM  *  0.2  +  ENGIM  *  0.4  +  SMIM  *  0.15  +  PRIM  *  0.4  +  INSTIM  *  0.3  +  SSIM  *  0.2  +  MMIM  *  0.1  +  Avoidance  Costs 

Pr  ogram  Costs 


Note  that  this  formula’s  numerator  reflects  the  notional  return  on  investment  of  each 
WBS  element’s  improved  effect  overall  (across  all  work  supported)  plus  the  avoidance 
costs  generated,  and  does  not  depend  on  the  WBS  element’s  funding  per  program.  This 
measure  is  very  subjective,  but  is  indeed  based  on  the  knowledge  of  many  experts.  In 
reality,  the  notional  benefit  realized  ultimately  depends  on  the  world  environment, 
workload,  and  specific  value-adding  opportunities  presented  in  a  given  year.  However, 
as  a  first  approximation  of  value-added,  or  net  benefit,  the  Invested  Multiplier  can  be 
estimated  based  on  the  investment  in  the  tool  being  leveraged. 

The  Invested  Multiplier  for  each  WBS  category  is  derived  for  insertion  into  the 
JBG  NROI  fonnula.  Each  Invested  Multiplier  is  detennined  as  follows: 


WBS  Category  Invested  Multiplier  =  Supporting  Tool  Investment  *  Invested  Im  pact  Factor 

Objectively,  we  can  determine  the  Program  Investment,  which  is  also  used  in  the 
denominator  as  Program  Costs.  Subjectively,  we  must  determine  the  expected  leveraged 
impact  and  construct  an  associated  factor.  For  example,  for  every  dollar  invested  in 
NDEX,  if  we  expect  that  fifty-cents  ($0.50)  of  resulting  benefit  to  program  management 
at  SPAWAR  exists,  a  factor  of  0.5  would  result.  This  would  encompass  benefits  such  as 
time  invested  in  managing,  better  informed  decision-making  and  better  acquisition 
decisions.  In  application,  we  chose  to  be  conservative,  but  given  the  amount  of  dollars 
invested  in  program  management  at  SPAWAR,  the  value  of  having  NDEX  to  a  particular 
decision  could  be  as  much  as  20  times  the  investment,  or  more.  However,  to  project  such 
a  realized  return  would  be  haphazard  and  insulting  at  best.  Therefore,  in  application,  we 
recommend  applying  the  factors  on  scale  with  the  investment  to  yield  no  more  than  a  1.0 
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return  to  any  WBS  element  if  greater  benefit  cannot  be  supported  empirically  (which  we 
did).  In  measuring  the  impact  over  time  once  a  program  is  implemented,  however,  the 
factor  may  be  empirically  supported  to  be  much  greater  than  the  magnitude  of  the 
investment. 
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IV.  APPLICATION  OF  THE  METHODOLOGY 


A.  ESTIMATING  THE  COST  OF  IMPLEMENTING  NDEX 

NDEX  will  not  be  a  very  expensive  project  for  the  Navy  to  implement.  The 
majority  of  the  equipment  that  is  used  by  NDEX  is  commercial  off  the  shelf  (COTS), 
subsequently;  items  are  readily  available  from  commercial  sources  and  will  not  require 
further  development. 

The  majority  of  the  costs  associated  with  NDEX  are  with  hiring  new  people  to 
perform  the  functions  of  NDEX.  As  it  stands,  NDEX  will  have  offices  on  both  the  East 
Coast  and  West  Coast  in  large  fleet  concentration  areas  in  order  to  handle  the  bulk  of 
ships.  It  is  initially  proposed  to  have  five  CAD  operators  on  each  coast  to  work  with  the 
ICD  and  class  design  baseline  drawings.  Furthermore,  it  will  be  required  to  hire  two 
programmers  and  two  engineers  for  each  coast  to  perform  work  with  network  analysis 
and  CADIT  programming.  The  total  anticipated  cost  of  all  combined  labor  is  expected  to 
be  $2.5M. 

Bringing  NDEX  to  fruition  will  require  the  purchase  of  computer  hardware, 
software,  ancillary  support  equipment  needed  for  everyday  use,  training,  and  travel. 
Costs  as  described  below  were  contractor  provided  and  are  assumed  accurate  for 
implementation. 

Hardware  is  comprised  of  all  computer  support  to  include  laptop  computers  for 
personnel  utilizing  NDEX.  Software  includes  AutoCAD  and  the  necessary  collaborative 
web-based  tools.  NDEX  will  require  special  equipment  to  include  servers  and  a  T1 
internet  line.  Of  note  is  that  all  hardware  and  software  is  based  on  a  two  year  life  cycle. 
If  equipment  malfunctions  or  becomes  obsolete  earlier  than  the  anticipated  two  year  life 
cycle,  costs  will  likely  increase. 

There  are  many  everyday  consumable  items  that  will  be  required  for  sustained 
use.  These  include  items  such  as  printers,  print  cartridges,  laptop  stands,  thumb  drives, 
and  other  items  necessary  to  carry  out  the  day  to  day  operations.  These  items  have  a  life 
cycle  that  varies  from  one  to  four  years. 
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Since  NDEX  is  a  new  system  which  incorporates  new  ideas  and  a  new  approach 
to  design  work,  professional  training  will  need  to  be  conducted.  Training  is  offered 
through  Autodesk,  the  developer  of  AutoCAD  and  is  currently  scheduled  every  two  years 
as  either  a  refresh  for  existing  employees  or  initial  training  for  new  hires. 

Travel  is  listed  as  an  expense  but  is  truly  a  negligible  expense  due  to  the  fact  that 
travel  will  be  necessary  whether  NDEX  is  implemented  or  the  current  system  is  utilized. 
There  will  still  need  to  be  site  visits  and  ship  check  regardless  of  the  system  in  place.  For 
the  sake  of  this  project,  travel  expenses  were  factored  in  to  the  NROI  equation  as  they 
will  occur  as  part  of  the  implementation  of  NDEX.  Furthermore,  travel  expenses  are 
expected  to  reduce  as  fewer  trips  will  need  to  be  made  and  fewer  people  will  be  required 
to  travel  due  to  the  efficiencies  of  NDEX.  Of  note  is  that  if  NDEX  expands  beyond  the 
C4I  suite  applications  and  into  complete  ship  design,  additional  travel  will  be  necessary 
but  costs  savings  will  increase  even  more  dramatically.  Travel  as  it  currently  stands  is 
expected  to  range  from  approximately  $53,000  during  NDEX’s  first  year  of  operation 
and  will  grow  by  about  $3,000  per  year  as  NDEX  expands.  (Dalton,  spreadsheet) 


Table  3  shows  expected  start  up  expenses  for  NDEX  starting  in  fiscal  year  2006 


and  going  through  fiscal  year  2010.  Other  direct  costs  (ODC)  include  all  hardware, 
software,  and  peripheral  support  equipment. 


TOTAL  LABOR 
TOTAL  TRAVEL 

TOTAL  ODC 
TOTAL  ANNUAL 
COST 


Table  3.  NDEX 
FY06 

$2,530,258 

$53,314 

$43,529 


Start-up  costs  per  fiscal  year 


FY07 

$2,656,771 

$55,980 

$45,705 


FY08 

$2,789,609 

$58,779 

$47,991 


FY09 

$2,929,090 

$61,718 

$50,390 


$2,627,101 


$2,758,456  $2,896,379  $3,041,198 


FY10 

$3,075,544 

$64,804 

$52,910 

$3,193,258 


B.  ESTIMATING  THE  BENEFITS  OF  IMPLEMENTING  NDEX 

There  are  many  potential  benefits  that  may  result  from  establishing  NDEX  as  a 
Navy  standard  for  coordinating  the  design  efforts  related  to  ship  installations.  These 
include  but  are  not  limited  to  cost  reductions,  technological  advance,  data-sharing,  and 
three  delivery  benefits:  end-user  value,  worldwide  availability,  and  contingency 
capability. 
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Significant  cost  reductions  are  expected  due  to  several  work  processes  changing 
and  a  decrease  in  time  required  for  completing  installations.  With  laptops  and  CAD 
operators  on-site  with  the  engineer,  the  design  teams  will  be  more  streamlined  and 
effective  in  integrating  ICD’s  and  completing  SID  packages.  Duplication  of  effort  will 
be  nearly  eliminated.  Information  will  be  readily  available  to  interested  parties;  thereby 
decreasing  the  amount  of  time  spent  drafting  redundant  drawings.  In  addition, 
management  decisions  can  be  made  more  quickly  based  on  a  single  available  network  of 
information  servers  which  will  save  time  throughout  the  installation  and  in  numerous 
subordinate  or  supporting  processes  such  as  shipyard  bidding  and  engineering  redesigns. 
Finally,  design  and  design-related  installation  Change  Order  Request  Notifications 
(CORNs)  will  be  reduced  and  changes  made  will  be  less  expensive  to  process.  Figure  1 
summarizes  the  expected  cost  avoidances  as  they  relate  to  the  current  design  process. 
The  authors  assume  that  NDEX  will  be  phased  in  and  therefore  do  not  take  all  avoidances 
at  once.  In  addition,  avoidances  once  taken,  are  not  repeatedly  taken,  thus  a  diminishing 
return  of  avoidances  persists  in  the  model.  The  avoidances  are  taken  over  a  five  year 
period  based  on  an  assumed  schedule  that  reflects  both  the  phase-in  of  NDEX  effects  and 
the  diminishing  availability  of  avoidances  for  harvesting.  The  assumed  percentages  for 
years  two  through  six  are  set  at  15%,  25%,  35%,  15%,  and  10%.  These  should  be 
adjusted  in  future  models  should  further  data  become  available  concerning  the  phase-in 
of  NDEX.  Finally,  we  assume  25%  of  CORNs  avoidance  will  recur  each  year  due  to  the 
nature  of  designing  new  systems  (these  are  not  avoidable  altogether),  and  the  resulting 
value  of  $0.3 1M  represents  decreased  costs  associated  with  making  or  supporting  the 
required  changes  that  occur  similarly  each  year. 
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Figure  1 .  Cost  Avoidances  Generated  by  NDEX  Reflected  in  the  Current  Process  (Chart 

adapted  from  original  SPAWAR  briefing  slide) 


DESIGN  PRODUCT  & 
PROCESS  FLOW  COST  SAVINGS 


Develop  ICD:  1.5M  avoidance 
(30%  reduction  on  249  ICDs 
20K  avg  cost) 
Integrate  ICD:  1.245M 
avoidance  (249  @  5K  each) 


■  Program/System  Info 

■  AFOM  Data 

■  System  Life  Cycle  Cos^ 

■  Fielding  Plan 


Authorized 
Preliminary  Engineering 


TALLATION  DESIGN  LIFE  CYCLE 


Design  Work 


SCD  Phase  II 
Concept  Design 


'■  Block  Diagram 

•  Outline  &  Install  Drawl 

•  Cable  Run  Sheets 

•  Install  Material  List 

•  Drawing  List 


ShipcheckEsL  •  Shipcheck  Report 
ig  •  SID  Est  -  Space  Requirement 

SRD.  COP.  SSR  Est  -Power 
-HVAC 

—  Port  Availability 


•  System  Concept 
Development 

■Alt  Considerations 

•  ILS  Impact 

•  System  Life  Cycle 
Cost  Refinement 


Authorized 
Design  Development 


SCD  Phase  III 
Design  Development 


•  System  Drawings 

•  Arrangement  Drawings 

•  Structural  Drawings 

•  Removal  Drawings 

•  Cabling  Sheets 


Design  Work  (DSA  75%):  1/3  of 
deliverables  impacted  @  40% 
avoidance  =  $1.54M  (15%) 
Design  Work  (DSA  25%):  40% 
avoidance  =  6.16M 


DSA  during 
Install  Phase 


•COP 
•COSAL 
•SSR 
• CSOSS 

•  DC  Plates 

•  OSR 


Design  Changes 


$  Authorized 

Procurement,  Install  &  DSA 


SCD  impact: 
Captured  only  in  Prog 
Mgmt/Eng/Prod 
Notional  Benefit 


•  Design  Development 

-Shock  test 
-  Prototype 
-DT/OT 

•  Alteration  info 

•  System  Life  Cost  Refinement 


•Impact  to  Cost,^ 
Schedule  &  Quant) 
•GFI  I 

•  Shipcheck  Issues 
•Tasking 


.69M  Design  CORN  avoidance 
.55M  Design-related  Install  CORN  avoidance 
Future  Year  Estimate  25%  each  year,  or 
.31  M/year 


T 


LAR/RLAR 


•Technical  Information 

•  Interpretation  of  drawings  specifications 

•  Material  identification 

•  Change  request 

•  Planning  Yard  approval  of  drawing  changes 


As  outlined  previously,  several  initiatives  in  our  business  environment  dictate  that 
we  become  more  and  more  network-centric  and  time  efficient  in  our  processes.  NDEX 
aligns  the  installation  process  with  other  21st  century  business,  technology,  and  defense 
processes.  This  will  improve  speed,  reliability  and  coordination  through  utilizing 
network-centric  technology  which  will  automate,  eliminate  redundancies,  and  enable 
NDEX  to  support  and  integrate  with  other  modem  systems. 

A  single  network  for  data-sharing  will  also  prove  beneficial  in  at  least  two  ways. 
First,  data-sharing  will  create  value  through  increased  corporate  knowledge.  In  the 
shipbuilding  industry,  where  expertise  is  growing  short,  the  movement  toward  utilizing  a 
reliable  database  system  will  likely  capture  and  protect  the  knowledge  developed  over 
many  years  and  hopefully  integrate  automated  skills  and  knowledge  which  may  be  in 
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short  supply  in  the  future.  Second,  data-sharing  will  level  the  playing  field  in  the 
contracting  and  bidding  process,  hopefully  returning  value  to  the  Navy  and  contractors  by 
providing  a  greater  amount  of  precise  specifications  needed  for  cost  estimation.  This 
should  drive  contract  prices  down. 

Finally,  and  perhaps  most  significantly,  NDEX  is  expected  to  improve  delivery  of 
technology  to  the  fleet  and  vastly  improve  the  capability  of  the  design  process  to  respond 
if  necessary  to  any  contingency  requirement.  While  delivering  this  technology,  the 
NDEX  process  should  be  far  less  imposing  on  the  end-user.  This  convenience  should  not 
be  underestimated,  as  it  reduces  the  ships’  own  investment  of  time  and  resources  in  the 
process.  These  savings  can  be  leveraged  at  improving  training,  maintenance,  or  any 
number  of  other  value-adding  activities. 

The  ability  to  handle  multiple  installations  in  a  short  timeframe  provides  needed 
capability  to  install  urgent  and  routine  system  requirements  fleet-wide  if  necessary.  In 
addition,  design  can  be  conducted  worldwide  and  managed  through  the  network  for 
approval,  allowing  installations  to  be  done  even  while  underway  if  necessary.  As  stated, 
NDEX  will  allow  the  Ship  Change  Document  (SCD)  to  be  transmitted  and  approved 
while  still  on  the  site-visit  if  necessary  to  facilitate  installations.  Therefore,  ships 
overseas,  especially  those  subject  to  “Sea  Swap”  that  will  remain  overseas  for  significant 
periods  of  time  and  be  supported  by  rotating  crews,  will  be  equally  supported  without 
multiple  travel  requirements  or  a  return  to  CONUS.  This  allows  greater  flexibility  and 
support  of  the  fleet  during  the  design  and  installation  process. 

How  are  these  benefits  included  and  measured  in  the  formula?  They  are  captured 
in  the  Invested  Multipliers.  As  previously  stated,  NDEX  impacts  the  current  work 
processes  by  providing  better  data  and  efficient  support.  The  WBS  structure  captures 
program  costs  for  processes  that  add  value  to  the  end  product.  NDEX  improves  these 
values  across  all  affected  programs. 

WBS  categories  and  their  associated  value  added  factors  (VAFs)  were  derived  by 
experts  and  reported  in  the  JBG  NROI  original  paper  (Bigham  30).  The  NDEX  Invested 
Multipliers  for  each  category  are  derived  from  the  authors’  expected  value  added  to  these 
processes  through  the  use  of  NDEX,  and  are  phased  in  as  NDEX  matures  at  the  following 
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subjective  rates  corresponding  to  years  of  maturity:  10%,  25%,  50%,  75%,  100%.  For 
example,  an  Invested  Multiplier  of  0.5  at  full  maturity  would  provide  0.05  times  the 
NDEX  investment  in  return  in  the  first  benefit  year.  The  resulting  dollar  value  would 
then  be  multiplied  by  the  WBS  category’s  VAF  to  determine  the  overall  WBS  category’s 
benefit  resulting  from  NDEX  in  that  year.  These  results  are  summarized  in  Table  4. 

The  following  explanations  are  provided  for  each  WBS  category’s  Invested 
Multiplier. 

Program  Management  /  Invested  Multiplier  (PMIM):  A  weighting  factor  of 
0.1  was  applied  to  program  management  in  the  JBG  NROI  model,  based  on  the  agreed 
amount  of  value  added  to  dollars  spent  within  this  category  (Bigham  33).  Program 
managers  were  perceived  to  have  the  education  and  experience  necessary  for  analyzing 
decisions  within  their  programs,  and  benefit  from  discussing  and  sharing  issues  with  each 
other  in  a  collaborative  environment.  A  number  of  project  management  tools  are  used 
by  program  management  to  add  value  to  their  programs.  NDEX  will  improve  the  design 
process  and  support  collaboration,  decreasing  the  time  and  effort  required  for  acquiring 
design  data  critical  to  program  decisions,  and  adding  value  by  shortening  the  design  cycle 
time,  enabling  program  managers  to  deliver  end-products  quicker.  The  NDEX  Program 
Management  Invested  Multiplier  (PMIM)  at  NDEX’s  full  maturity  is  0.2,  meaning  for 
every  dollar  spend  on  NDEX,  the  NDEX  tool  will  return  a  total  value  of  20  cents  to 
supported  program  management  decisions. 

Engineering  (ENG):  A  weighting  factor  of  0.4  was  applied  to  engineering  in  the 
JBG  NROI  model,  based  on  the  agreed  amount  of  value  added  to  dollars  spent  within  this 
category  (Bigham  33).  Harvesting  past  experience  in  developing  engineering  products 
for  a  number  of  deployed  systems  is  the  primary  value-adding  process  in  Engineering. 
NDEX  will  provide  better  knowledge  capture  related  to  completed  designs,  improve  the 
ability  to  leverage  engineering  products  from  one  hull  to  the  next,  and  provide  Class 
Design  Baseline  management  to  vastly  improve  efficiency  for  integrating  Installation 
Control  Designs  (ICDs),  enabling  faster  integration  of  technology  in  the  fleet.  The  NDEX 
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Engineering  Invested  Multiplier  (ENGIM)  at  NDEX’s  full  maturity  is  0.4,  meaning  for 
every  dollar  spend  on  NDEX,  the  NDEX  tool  will  return  a  total  value  of  40  cents  to 
supported  Engineering  products. 

For  example,  once  a  specific  ICD  drawing  is  developed  for  a  given 
communication  system  (i.e.,  HF  system,  VHF/UHF  system,  etc),  it  becomes  the 
automated  design  starting  point  for  follow-on  system  installs  on  other  platfonns. 
NDEX’s  CDBs  provide  follow-on  platform  requirements.  NDEX  then  can  be  used  to 
provide  automated  integration  of  drawings  from  one  platform  to  the  next.  Leveraged 
across  multiple  hulls,  the  ICDs,  cable  run  sheets,  and  “As  Built”  drawings  are  captured 
and  remain  available  for  future  designers  and  installers.  Improved  technologies  can  thus 
be  employed  faster  in  the  fleet  and  the  feedback  loop  closed  very  quickly  on  engineering 
changes  or  design  modifications  required. 

Material  Management  (MM):  A  weighting  factor  of  0. 1  was  applied  to  material 
management  in  the  JBG  NROI  model  based  on  the  agreed  amount  of  value  added  to 
dollars  spent  within  this  category  (Bigham  33).  NDEX  improves  the  capacity  of  support 
for  spiral  acquisition  products  and  the  time-to-market  delivery  of  material  by  providing 
design-related  infonnation  sooner.  NDEX  may  also  improve  work  practices  by  easing 
access  to  needed  data.  Life  cycle  costs  of  material  support  may  be  reduced  as  well  by  the 
improvement  of  the  initial  design  and  supported  decisions  related  to  spares.  The  NDEX 
Material  Management  Invested  Multiplier  (MMIM)  at  NDEX’s  full  maturity  is  0.1, 
meaning  for  every  dollar  spend  on  NDEX,  the  NDEX  tool  will  return  a  total  value  of  10 
cents  to  supported  Material  Management  decisions. 

Supportability  Management  (SM):  A  weighting  factor  of  0.15  was  applied  to 

supportability  management  in  the  JBG  NROI  model  based  on  the  agreed  amount  of  value 

added  to  dollars  spent  within  this  category  (Bigham  33).  Time  available  for  crew 

familiarization  and  maintenance  training  will  increase  for  any  specific  platform  due  to  an 

improved  design  cycle  time.  Development  of  supporting  documents  and  classroom 

packages  will  improve  as  rework  is  reduced  by  reducing  drastic  design  changes  and  work 

stoppages.  Changes  to  existing  training  programs  will  be  easier  to  complete  with  readily 

available  design  data.  Sharing  of  lessons  learned  across  multiple  platfonns  may  also  be 
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incorporated  into  future  designs  in  a  very  quick  feedback  loop.  The  NDEX  Supportability 
Management  Invested  Multiplier  (MMIM)  at  NDEX’s  full  maturity  is  0.3,  meaning  for 
every  dollar  spend  on  NDEX,  the  NDEX  tool  will  return  a  total  value  of  30  cents  to 
supported  Supportability  Management  decisions. 

Production/Integration  (PR):  A  weighting  factor  of  0.4  was  applied  to 
production/integration  in  the  JBG  NROI  model  based  on  the  agreed  amount  of  value 
added  to  dollars  spent  within  this  category  (Bigham  33).  There  is  significant  “value 
added”  in  the  integration  and  testing  that  NDEX  supports.  NDEX  allows  for  automated 
integration  of  tested  components  with  existing  ICDs  into  the  CDB  within  hours. 
Production  efforts  like  providing  the  cables,  connectors,  back  shells  and  other  systems 
not  sponsored  by  hardware  providers  but  required  for  delivery  of  the  product  can  be 
planned,  scheduled,  and  completed  with  the  use  of  readily  available  and  timely  design 
data.  Interoperability  with  other  new  and  legacy  systems  in  the  design  process  can  be 
checked  prior  to  installation.  A  full  design  “mock  up”  of  the  shipboard  spaces  enables 
validation  of  the  proposed  design  configurations,  providing  improved  value  to  the  end 
user  who  avoids  delays  in  delivery.  The  NDEX  Production/Integration  Invested 
Multiplier  (PRIM)  at  NDEX’s  full  maturity  is  0.5,  meaning  for  every  dollar  spent  on 
NDEX,  the  NDEX  tool  will  return  a  total  value  of  50  cents  to  supported  Production  and 
Integration  processes. 

Installation  (INST):  A  weighting  factor  of  0.3  was  applied  to  installation  in  the 
JBG  NROI  model  based  on  the  agreed  amount  of  value  added  to  dollars  spent  within  this 
category  (Bigham  33).  NDEX  can  improve  the  installation  and  testing  time  needed  by 
validating  the  design  of  the  ICD  to  the  CDB  much  quicker  than  the  current  process.  This 
greatly  reduces  the  install  time  as  well  as  the  confusion  and  troubleshooting  related  to 
installations  within  the  industrial  environment  at  the  shipyard.  This  also  enables  delivery 
of  systems  that  are  closer  to  “state  of  the  art”  at  delivery.  Additional  benefits  of  these 
improvements  include  capturing  lessons  learned  quickly  and  incorporating  them  into 
future  installs,  as  well  as  improved  customer  satisfaction  with  the  installation  process. 
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The  NDEX  Installation  Invested  Multiplier  (INSTIM)  at  NDEX’s  full  maturity  is  0.6, 
meaning  for  every  dollar  spent  on  NDEX,  the  NDEX  tool  will  return  a  total  value  of  60 
cents  to  supported  Installation  processes. 

Shipyard  Support  (SS):  A  weighting  factor  of  0.2  was  applied  to  shipyard 
support  in  the  JBG  NROI  model  based  on  the  agreed  amount  of  value  added  to  dollars 
spent  within  this  category  (Bigham  33).  Civil  service  and  contractor  personnel  interface 
daily  with  the  shipbuilder  and  planning  yards.  NDEX  improves  communication  between 
PEO/SPAWAR  and  these  collaborators,  allowing  for  quicker  identification  and  resolution 
of  issues  and  concerns.  Design-related  issues  leading  to  stopped  work  orders  can  be 
extremely  expensive.  By  improving  the  design-related  communications  between  all 
parties  in  a  collaborative  data  environment  for  the  sharing  of  information,  NDEX  will 
reduce  the  circumstances  that  often  lead  to  stopped  work.  In  addition,  oversight  will  be 
improved  with  the  accessibility  and  control  of  design  information.  Feedback  loops  will 
be  improved  as  well.  The  NDEX  Installation  Invested  Multiplier  (INSTIM)  at  NDEX’s 
full  maturity  is  0.3,  meaning  for  every  dollar  spent  on  NDEX,  the  NDEX  tool  will  return 
a  total  value  of  30  cents  to  supported  Installation  processes. 


C.  ESTIMATING  THE  JBG  NROI  FOR  NDEX 

To  complete  the  estimation  of  the  JBG  NROI  for  the  NDEX  system,  a  spreadsheet 
model  (Table  4)  of  cost  and  benefit  data  was  developed  for  a  five-year  period  using  the 
factors  developed.  Benefits  are  assumed  to  lag  one  year  later  than  NDEX  funding,  or  be 
realized  in  the  Fiscal  Year  following  funding.  In  addition,  cost  avoidances  are  only 
reported  in  the  year  they  are  projected  to  be  realized,  not  in  subsequent  years.  Thus,  the 
cost  avoidances  in  aggregate  are  a  measure  of  real  better-off- ”ness”  in  terms  of  budget 
expense  at  the  end  of  the  period.  One  limitation  of  this  analysis  is  that  the  phasing-in  of 
NDEX  must  be  assumed-  therefore  subjective  percentages  are  based  simply  on  a 
reasonable  impact  schedule,  rather  than  an  available  schedule  or  data.  As  stated  above, 
the  percentages  chosen  for  phase-in  of  notional  benefits  by  year  were  10%  (second  year), 
25%  (third  year),  50%  (fourth  year),  75%  (fifth  year),  and  100%  (sixth  year).  The 
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percentages  chosen  for  phase-in  of  avoidances  by  year  were  15%  (second  year),  25% 
(third  year),  35%  (fourth  year),  15%  (fifth  year),  and  10%  (sixth  year). 

NDEX,  as  measured,  returned  a  195.94%  JBG  NROI.  Two  required  calculations 
are  demonstrated  below  for  understanding.  In  addition  to  the  calculations  made  below, 
the  net  present  values  of  future  year  Total  Benefits  were  computed  and  the  cumulative 
result  used  as  the  numerator  in  the  JBG  NROI  formula.  The  denominator  is  simply  the 
first  year  investment  in  NDEX.  While  the  numbers  produced  by  the  spreadsheet  model 
imply  significance  to  the  penny,  it  should  not  be  viewed  with  such  precision.  The 
numbers  are  provided  for  review  as  calculated,  but  should  be  rounded  appropriately 
should  the  answer  provided  be  used  in  further  reporting  (the  JBG  NROI  is  about  200% 
would  be  a  better  stated  approximation). 

The  following  is  the  computation  of  the  Program  Management  Invested  Multiplier 
(Year  2): 

Program  Management  ( PM )  Invested  Multiplier  (Year  2)  =  NDEX  Investment  Year  1  * 

PM  Investment  Im  pact  Factor  Year  2 

PMIM  =  $2,627,101  *  0.02  =  $52,542.02 

The  following  is  the  computation  of  the  Program  Management  Notional  Benefit  from 
NDEX  (Year  2): 

PM  Notional  Benefit  ( Year  2)  =  PMIM  (Year  2)  *  PM  Value  Added  Factor  ( VAF ) 

PM  Notional  Benefit  (Year  2)  =  $52,542.02  *  0.1  =  $5,254.20 


After  calculating  the  notional  benefit  for  each  WBS  category  in  each  of  the  5  benefit 
years  (years  two  through  six),  the  sum  of  the  benefits  in  all  WBS  categories  for  each  year 
is  calculated.  We  arrive  at  our  numerator  by  then  adding  the  total  avoided  costs  to  the 
notional  benefits  in  each  benefit  year  and  subtracting  any  new  investment  in  NDEX. 
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Then,  each  year’s  total  benefit  is  valued  at  Net  Present  Value  and  summed  to  arrive  at  the 
numerator  value  of  $5,147,508.22.  The  final  calculation  of  the  JBG  NROI  (modified  for 
NDEX)  is 


JBG  NROI 


PMIM  *  0.2  +  ENGIM  *  0.4  +  SMIM  *  0.15  +  PRIM  *  0.4  +  INSTIM  *  0.3  +  SSIM  *  0.2  +  MMIM  *  0.1  +  Avoidance  Costs 

Pr  ogram  Costs 


JBG  NROI  = 


$5,147,508.22  (Summation  of  all  five  years  at  NPV) 
$2,627, 101.00  (First  year  investment  in  NDEX) 


=  195.94%  or  200% 
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Figure  2.  NDEX  NROI  Computation 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS  DRAWN  FROM 

APPLICATION 

The  purpose  of  this  project  was  to  provide  an  ROI  computation  for  NDEX.  With 
much  thought  and  after  carefully  studying  all  public  sector  ROI  computations,  it  was 
determined  that  the  best  course  of  action  in  applying  the  JBG  NROI  formula  to  this 
application  was  by  valuing  the  notional  benefits  and  avoidances  created  by  the 
investment  in  NDEX  based  on  the  Work  Breakdown  Structure  of  SPAWAR  programs  in 
order  to  detennine  the  benefit  NDEX  generates  when  infused  into  the  current  process. 
The  resulting  JBG  NROI  (modified)  of  195.94%  (or  about  200%  rounded),  represents  a 
significant  opportunity  for  investment.  This  investment  would,  if  completed,  provide 
excellent  return  and  modernize  the  design  process  SPAWAR  currently  utilizes. 

Based  on  the  information  provided  in  this  document,  it  appears  that  implementing 
NDEX  would  provide  substantial  cost  avoidance  in  addition  to  added  benefit  over  many 
years.  Even  with  the  initial  startup  costs  associated  with  bringing  a  new  process  online, 
the  savings  over  time  would  be  tremendous.  Calculations  based  on  the  modified  JBG 
NROI  computation  for  NDEX  show  that  in  a  short  time  the  budget  could  absorb  NDEX 
funding  completely  and  realize  additional  savings  and  benefits.  However,  in  reality,  the 
result  depends  on  solid  implementation  and  the  ability  to  break  existing  paradigms  in 
order  to  maximize  efficiency,  effectiveness,  and  savings. 

Implementing  NDEX  will  not  be  an  easy  task.  It  will  require  substantial  buy-in 
from  all  levels  ranging  from  the  planning  yards  to  program  managers  to  NDEX  operators, 
and  all  the  way  to  the  ships  the  process  supports.  It  will  also  require  the  organizations 
who  deal  with  ship  design  applications  to  perhaps  modify  their  modus  operandi. 
Additionally,  there  may  be  cultural  resistance  from  those  who  were  doing  the  job  the  old 
way.  They  may  feel  that  the  current  system  is  not  broken  and  therefore  should  continue 
unaltered.  Therefore,  it  is  again  recommended  that  existing  paradigms  be  carefully 
considered  and  challenged  in  order  to  realize  the  potential  benefits  NDEX  might  provide. 

Limitations  do  exist,  however.  First,  the  Invested  Multipliers  used  to  determine 
the  added  value  NDEX  provides  to  programs  must  be  refined  and  verified  by  program 
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experts.  Prototyping  utilization  of  NDEX  will  provide  data  related  to  existing  programs 
that  may  be  used  to  enhance  the  model  provided.  This  may  also  require  a  survey  and 
research  beyond  the  scope  of  this  project.  Additionally,  the  Invested  Multiplier  concept 
can  only  be  applied  toward  a  tool  or  program  that  impacts  other  programs.  Finally,  the 
factors  used  for  calculating  the  Invested  Multiplier  are  subjective  and  hard  to  estimate. 
However,  we  do  believe  that  when  the  tool  investment  is  substantially  less  than  the 
program  investments  affected,  as  in  the  case  of  NDEX,  that  reasonable  notional  benefits 
based  on  the  tool  investment  and  a  reasonable  approximation  of  factors  which  produce 
Invested  Multipliers  will  provide  a  very  good  first  approximation  of  the  value  of  the  tool 
which  may  be  used  for  evaluating  the  tool’s  notional  return  on  investment.  The  resulting 
notional  return  on  investment  may  then  be  used  in  the  decision  process  as  one  of  many 
data  points  in  a  go/no-go  decision  to  implement  the  tool. 

Several  benefits  of  this  research  have  been  realized.  One  benefit  of  the  model  is 
that  the  scale  may  be  modified  through  the  adjustment  of  the  factors  to  value  a  tool  like 
NDEX  to  one  or  many  programs.  The  model  is  also  adaptable  as  more  data  becomes 
available.  This  flexibility  enables  the  model  to  be  used  for  evaluating  numerous 
investments.  In  addition,  there  is  great  utility  in  the  concept  of  valuing  a  process  such  as 
NDEX  based  on  its  ability  to  improve  the  value  a  program  or  many  programs  may  deliver 
to  the  end-user  as  it  relates  to  the  Work  Breakdown  Structure.  This  method  puts  notional 
benefits  in  a  language  common  to  the  Department  of  Defense,  and  enables  capture  of 
both  potential  benefit  of  the  new  process  or  product  and  realized  benefit  based  on  current 
program  capabilities  to  deliver  value  in  the  first  place.  The  model  ultimately  measures 
NROI  from  the  perspective  of  the  stakeholder’s  improved  capability  of  delivering  end- 
user  value. 

Finally,  the  authors  realize  the  value  of  a  notional  return  on  investment 
calculation  should  never  be  fully  authoritative  or  prohibitive  in  making  decisions.  In  the 
case  of  NDEX,  it  is  important  to  realize  that  recent  business  trends  internationally  and  in 
the  public  sector  support  the  decision  to  implement,  as  does  technological  awareness 
which  demands  design  process  improvement.  It  appears  obvious,  based  on  our  research 
of  analogous  projects  and  in  our  conversations  with  all  parties,  to  implement  design 
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process  improvement  throughout  the  Department  of  the  Navy.  This  is  necessary  for 
driving  cost  behavior  and  valuable  to  existing  programs,  and  a  return  is  virtually  assured. 
Furthermore,  as  a  first  mover,  SPAWAR  could  realize  a  significant  return  on  its  own 
investment  in  design  process  modernization  if  the  friction  encountered  in  modernizing 
can  be  overcome.  Ultimately,  implementation  and  the  ability  to  reengineer  current 
processes  will  be  the  key  to  NDEX’s  success  or  failure. 
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